Predictive and profile learning sales automation analytics system and method

ABSTRACT

A sales automation system and method, namely a system and method for scoring sales representative performance and forecasting future sales representative performance. These scoring and forecasting techniques can apply to a sales representative monitoring his own performance, comparing himself to others within the organization (or even between organizations using methods described in application), contemplating which job duties are falling behind and which are ahead of schedule, and numerous other related activities. Similarly, with the sales representative providing a full set of performance data, the system is in a position to aid a sales manager identify which sales representatives are behind others and why, as well as help with resource planning should requirements, such as quotas or staffing, change.

CROSS-REFERENCES TO RELATED APPLICATIONS

The present application is a Divisional of U.S. application Ser. No.13/617,437, filed on Dec. 28, 2006, entitled “PREDICTIVE AND PROFILELEARNING SALESPERSON PERFORMANCE SYSTEM AND METHOD,” of which the entiredisclosure is incorporated herein by reference for all purposes.

The present application is also related to the following co-pending andcommonly assigned U.S. patent applications:

U.S. patent application Ser. No. ______ (Attorney Docket Number88325-883762(125810US)) filed concurrent herewith and entitled“PREDICTIVE AND PROFILE LEARNING SALES AUTOMATION ANALYTICS SYSTEM ANDMETHOD,” and which claims priority to U.S. application Ser. No.13/617,437, filed on Dec. 28, 2006, entitled “PREDICTIVE AND PROFILELEARNING SALESPERSON PERFORMANCE SYSTEM AND METHOD;”U.S. patent application Ser. No. ______ (Attorney Docket Number88325-883766(125820US)) filed concurrent herewith and entitled“PREDICTIVE AND PROFILE LEARNING SALES AUTOMATION ANALYTICS SYSTEM ANDMETHOD,” and which claims priority to U.S. application Ser. No.13/617,437, filed on Dec. 28, 2006, entitled “PREDICTIVE AND PROFILELEARNING SALESPERSON PERFORMANCE SYSTEM AND METHOD;”U.S. patent application Ser. No. ______ (Attorney Docket Number88325-883767(125830US)) filed concurrent herewith and entitled“PREDICTIVE AND PROFILE LEARNING SALES AUTOMATION ANALYTICS SYSTEM ANDMETHOD,” and which claims priority to U.S. application Ser. No.13/617,437, filed on Dec. 28, 2006, entitled “PREDICTIVE AND PROFILELEARNING SALESPERSON PERFORMANCE SYSTEM AND METHOD;” andU.S. patent application Ser. No. ______ (Attorney Docket Number88325-883769(125840US)) filed concurrent herewith and entitled“PREDICTIVE AND PROFILE LEARNING SALES AUTOMATION ANALYTICS SYSTEM ANDMETHOD,” and which claims priority to U.S. application Ser. No.13/617,437, filed on Dec. 28, 2006, entitled “PREDICTIVE AND PROFILELEARNING SALESPERSON PERFORMANCE SYSTEM AND METHOD.”

BACKGROUND

1. Field

This application relates generally to sales automation and forecastingsystems and, more particularly, to tracking and storing sales data forperformance prediction.

2. Background

Sales automation systems are common in the art. In standard usage, thesesystems aid a sales representative, a sales manager, or both, to be moreefficient at selling a product or service. Sales forecasting analyticsare commonly a component of a sales automation systems, and are largelyfocused towards the sales manager to help forecast and manage the salesprocess.

A sales automation system is generally a tool that allows salesrepresentatives and managers to organize contact records, as well asmanage records associated with sales quotes and invoices. It can alsowork in the context of a ‘Sales Methodology’ where the sales process isstructured around the representative working though a set of stages inan attempt to complete a sale. Sales automation tools typically allowthe tracking of such records in terms of time periods associated withfiscal or financial accounting. During these ‘sale periods’ it isimportant for sales representatives and managers to have analyticalreports defining progress towards various goals.

Typically the sales representative recognizes the benefit of using asales automation system for maintaining their list of contacts, but mostother tasks are considered overhead with little benefit. Thus, salesautomation systems commonly suffer from a lack of full acceptance by thesales staff, which limits their usefulness to the sales managers aswell—if the sales representative doesn't utilize the system then themanager does not have a full picture of the sales process. As the salesrepresentative is responsible for using the system to track individualdeal progress through the sales stage, should they fail to log the salesprogress the system is left without the valuable data necessary for thesales manager to forecast and manage the sales goals. A system thatprovides incentive to a sales representative to use features beyond acontact management system is needed. Additionally, in part because ofthe lack of sales data and in part because of the prior state of theart, many analytics used by the sales manager are quite simple modelswith manual parameters based upon the sales manager's intuitions aboutprior performance.

Many algorithms in the general field of data mining provide resources toa knowledgeable individual for extracting relevant information fromlarge amounts of data. There exist data mining applications to aid inthis process. In some instances there are data mining approachesincorporated into other systems, such as sales automation systems. Theseincorporated techniques are usually quite rudimentary compared to thefull suite of techniques available in a complete data mining system, yetstill require some level of sophistication on the part of the user (insales automation systems the user would be the sales manager). Some ofthe more advanced techniques available would be standard statisticalapproaches for assigning error bars or applying a linear regressionanalysis. These statistical approaches are often guided by or overriddenwith ad-hoc scaling factors based upon the sales manager's intuition,such as: “Bob usually over promises his amount sold by about 25%, yetSue is more conservative and usually under promises by 15%. Therefore Iwill adjust Bob's sales predictions down by 25%, but increase Sue'spredictions by 15%.”

There exist new data mining and machine learning techniques which can gobeyond the traditional analyses, above, but they require data to workaccurately to overcome the ad-hoc manual scaling factors. To collectthis data the sales representative must be motivated to provide theinformation. Stereotypically, sales people are motivated by two goals:meeting personal monetary targets, and out performing their peers.Methods that target these motivations are needed and will increase theacceptance of a sales automation system by the sales people, and henceprovide a richer set of useful data to the sales manager. Most priorsystems have failed to adequately provide features found compelling tothe sales representative, and have overlooked the connection that thesales manager's job is best done with the full data available from anengaged sales representative. Some prior systems have recognized thebenefits of catering to the interests of a sales representative, buthave neglected to use the data naturally collected by the salesautomation system to reflect back and help provide the necessaryfeedback to keep the system accurately tuned—commonly the systems reliedon a manual configuration of the various parameters.

A better sales automation and forecasting management system is needed toaddress the above noted shortcomings in the prior art.

SUMMARY

One aspect of the inventive subject matter includes a sales automationsystem and a method for scoring sales representative performance andforecasting future sales representative performance. These scoring andforecasting techniques can apply to a sales representative monitoringhis own performance, comparing himself to others within the organization(or even between organizations using methods described in application),contemplating which job duties are falling behind and which are ahead ofschedule, and numerous other related activities. Similarly, with thesales representative providing a full set of performance data, thesystem is in a position to aid a sales manager identify which salesrepresentatives are behind others and why, as well as help with resourceplanning should requirements, such as quotas or staffing, change.

Further aspects include a central repository of logged sales relateddata including data representative of the progress made through thevarious sales stages of the sale cycle. The logged sales data isextracted to support the learning forecasting and prediction functionsof the present invention. Numerical and quantitative data records can beretrieved based on user request for data. Still further aspects includean incentive because of the ability to provide a sales representativethrough a user interface current and predictive analysis of theirperformance relative to their peers and groups of other salesrepresentatives. Some embodiments also provide a tool that allows thesales representative and manager to perform a predictive analysis oftheir conversion rate to monitor monetary performance.

For example, all calculations can be done to score the performance ofindividual people and groups with respect to learned data models of topperforming sales people as well as with respect to user-entered idealmodels of performance. An example of this would be an ideal revenuegrowth of a sales person over time. A second example is a relativeanalysis of the person's ongoing sales ‘pipeline’ with respect tolearned and ideal models of good performance. In either case the systemcan learn the historical sales models for an individual or highperforming sales representative using any of a variety of machinelearning techniques. In addition, an idealized model can be entered bythe sales manager. The resulting comparison can aid in the understandingof how individuals relate to one another and to an idealized individual.This helps sales representatives understand their relative performance,and it helps sales managers understand if there is a general deviationbetween their idealized model and the actual performance of anindividual or group.

Another example can be evaluating an entire sales process salesmethodology with respect to the ideal model of the process as well ascompared to other good performing processes or methodologies. Note thatthis analysis can be independent of individuals looking at the processin isolation. It can also be used to identify weak individuals or groupsthat bring the overall performance of the process or methodology down.One aspect of the embodiments is the ability to use the systems andmethods in CRM software applied to sales process analysis and itscombination with iterative machine learning methods.

With the historical performance of each sales representative stored bythe sales automation system, the system can more accurately update theforecasting portion of the sales analytics.

These and other advantageous features of the various embodiments will bein part apparent and in part pointed out herein below.

BRIEF DESCRIPTION OF THE DRAWINGS

For a better understanding of the present invention, reference may bemade to the accompanying drawings in which:

FIG. 1 is a top level diagram of the system architecture;

FIG. 2 is an illustration of the data agglomeration of the respectiveelements of the central repository;

FIG. 3 is an illustration of a data query module according toembodiments of the invention;

FIG. 4 is a machine learning and forecasting functional flow accordingto embodiments of the invention;

FIG. 5 is a future planning module functional flow according toembodiments of the invention;

FIGS. 6 and 7 are diagrams illustrating the revenue growth MLF examples;

FIG. 7 is an illustration of the Ideal Growth Analysis;

FIG. 8 is an illustration of the Ideal Pipeline or Strategic analysis;

FIG. 9 is an illustration of Query, Analytic Comparison and displaymethod;

FIG. 10 is an illustration of the graphical output of Ideal RevenueGrowth Analysis;

FIG. 11 is an illustration of the graphical output of Ideal Pipeline orStrategy Analysis;

FIG. 12 is a diagram of the Data Manipulation Algorithm (DMA);

FIG. 13 is a diagram of the DMF flow;

FIG. 14 is a diagram of the Post-DMF Heuristics algorithm;

FIGS. 15A and 15B are a flowchart illustrating a Pipeline Analysismethod according to embodiments of the invention;

FIGS. 16A and 16B are flowcharts illustrating a revenue forecastingmethod according to embodiments of the invention;

FIGS. 17A and 17B are a flowchart illustrating a method for scoringperformance according to embodiments of the invention;

FIG. 18 is a flowchart illustrating a method for allocation of quotas byperiods according to embodiments of the invention; and

FIG. 19 is a flowchart illustrating a method for allocation of quotas byterritories according to embodiments of the invention.

INCORPORATION BY REFERENCE

An addition to the disclosure herein relative to the subject matters A-Dreference is to be made to the publications listed below:

(A) Queuing Models/Systems:

-   An Introduction to Stochastic Modeling by Samuel Karlin, Howard M.    Taylor Academic Press; 3 edition (February 1998) (hereby    incorporated by reference for all purposes)

(B) Machine Learning & Data Mining

-   Pattern Recognition, Third Edition by Sergios Theodoridis,    Konstantinos Koutroumbas; Academic Press; 3 edition (Feb. 24, 2006)    (hereby incorporated by reference for all purposes)-   Pattern Classification (2nd Edition) by Richard O. Duda, Peter E.    Hart, David G. Stork; Wiley-Interscience; 2nd edition (October 2000)    (hereby incorporated by reference for all purposes)-   Machine Learning by Tom M. Mitchell; McGraw-Hill    Science/Engineering/Math; 1 edition (Mar. 1, 1997) (hereby    incorporated by reference for all purposes)

(C) Forecasting and Prediction:

-   Forecasting: Methods and Applications by Spyros G. Makridakis,    Steven C. Wheelwright, Rob J Hyndman; Wiley; 3 edition    (December 1997) (hereby incorporated by reference for all purposes)

(D) Sales/Business:

-   Selling and Sales Management by Robert D. Hirsch, Ralph Jackson;    Barron's Educational Series (Sep. 29, 1993) (hereby incorporated by    reference for all purposes)

DEFINITION OF TERMS

Sales Strategy: A sales methodology where the process of selling isorganized around a set of abstract ‘stages’, where at each stage thesales person and/or potential customer perform a set of tasks.

Sales Pipeline: A set of potential sales transactions (deals) for asales person or group in various stages of completion.

Sales Funnel: A standard visual picture of a sales pipeline.

Sales Period: A specific time period when sales activity is conductedand measured.

Sales Quota: A revenue goal for a particular sales person or group ofpeople for a specific ‘sales period’.

Top Performer: A sales person who meets a set of user/system definedperformance metrics for a set of sales periods.

Data Agglomeration: The process of grouping a set of raw data points bya particular attribute along with a particular mathematical operator.

DETAILED DESCRIPTION OF INVENTION

In the following detailed description of exemplary embodiments of theinvention, reference is made to the accompanying drawings that form apart hereof, and in which is shown by way of illustration specificexemplary embodiments in which the invention may be practiced. Theseembodiments are described in sufficient detail to enable those skilledin the art to practice the inventive subject matter, and it is to beunderstood that other embodiments may be utilized and that logical,mechanical, electrical and other changes may be made without departingfrom the scope of the inventive subject matter.

Some portions of the detailed descriptions which follow are presented interms of algorithms and symbolic representations of operations on databits within a computer memory. These algorithmic descriptions andrepresentations are the ways used by those skilled in the dataprocessing arts to most effectively convey the substance of their workto others skilled in the art. An algorithm is here, and generally,conceived to be a self-consistent sequence of steps leading to a desiredresult. The steps are those requiring physical manipulations of physicalquantities. Usually, though not necessarily, these quantities take theform of electrical or magnetic signals capable of being stored,transferred, combined, compared, and otherwise manipulated. It hasproven convenient at times, principally for reasons of common usage, torefer to these signals as bits, values, elements, symbols, characters,terms, numbers, or the like. It should be borne in mind, however, thatall of these and similar terms are to be associated with the appropriatephysical quantities and are merely convenient labels applied to thesequantities. Unless specifically stated otherwise as apparent from thefollowing discussions, terms such as “processing” or “computing” or“calculating” or “determining” or “displaying” or the like, refer to theaction and processes of a computer system, or similar computing device,that manipulates and transforms data represented as physical (e.g.,electronic) quantities within the computer system's registers andmemories into other data similarly represented as physical quantitieswithin the computer system memories or registers or other suchinformation storage, transmission or display devices.

In the Figures, the same reference number is used throughout to refer toan identical component that appears in multiple Figures. Signals andconnections may be referred to by the same reference number or label,and the actual meaning will be clear from its use in the context of thedescription. Also, please note that the first digit(s) of the referencenumber for a given item or part of the invention should correspond tothe Fig. number in which the item or part is first identified.

The description of the various embodiments is to be construed asexemplary only and does not describe every possible instance of theinventive subject matter. Numerous alternatives could be implemented,using combinations of current or future technologies, which would stillfall within the scope of the claims. The following detailed descriptionis, therefore, not to be taken in a limiting sense, and the scope of theinventive subject matter is defined only by the appended claims.

One embodiment of the present invention comprising a central repositoryof logged sales related data; a machine learning module and a predictionmodule teaches a novel system and method for sales performance andforecasting.

For simplicity and illustrative purposes, the principles of the presentinvention are described by referring mainly to an exemplar embodiment,particularly, with references to the Internet and the World Wide Web(WWW) as the exemplary databases of informational items. However, one ofordinary skill in the art would readily recognize that the sameprinciples are equally applicable to, and can be implemented in, otherinformational database, and that any such variation would be within suchmodifications that do not depart from the true spirit and scope of thepresent invention. Similarly, simplicity of description for the presentinvention makes use of a data warehouse. However, one of ordinary skillin the art would readily recognize that the same functionality can begained in the absence of a data warehouse through summarized data,simulated data, data models or the like and not depart from the truespirit and scope of the present invention.

FIG. 1 illustrates a representative architecture 100 according toembodiments of the invention. The details of the various embodiments canbe better understood by referring to the figures of the drawings. Thedata warehouse (DW) 102 is the central repository for the supporting themachine learning and forecasting (MLF) 108 and future planning (FP) 110modules. While those of ordinary skill in the art could readily identifydifferent sets or subsets of elements, for explanatory purposes in theembodiments of the present invention the following elements aredescribed as part of a data warehouse

-   -   Data Transforms—data    -   Heuristics {Idealized and Learned}—using an Idealized or Learned        set of rules or replicable method or approach for directing        attention to data key to prediction and forecasting    -   Data Models {Idealized and Learned}—using data models using        idealized or learning information    -   Derived Data Store    -   Data Cache    -   Metrics and Scores—related to sales goods    -   Stored Scenarios—fabricated sales conditions    -   Configuration Settings—

One aspect of the DW 102 of the various embodiments is that nearly allelements may have an associated timestamp. One of ordinary skill in theart can clearly see that the DW 102 can be maintained in a database, inworking memory, on disk, or by innumerable other means. Similarly, theDW 102 may contain original data, summarized data, machine learned data,or manually configured data. Referring to FIG. 1, the DW 102 is shownhaving the various elements listed above. The Data Agglomeration 104 isshown communicably linked to the DW 102 for retrieving and cachingrecords. The Query Module 106 for initiating user queries and displayingis shown communicably linked to the DW. The machine learning module 108and future planning module 110 is shown communicably linked to DW 102and Data agglomeration 104. The DW 102 includes sales data generatedfrom raw sales data manipulated by the MLF 108 and FP 110 to generatelearned data based on historical data, and includes idealized data basedon benchmarked models or user input or idealized data formulated fromdata operated on by heuristic rules/algorithms. A display and userinterface 116 may be used to access DW 102. Data provided through userinterface 116 may be transformed using run time heuristics and datatransforms 118. Additionally a user interface 114 may be used to provideinput and receive output from FP 110.

FIG. 2 is an illustration of the operation of data agglomeration module104 according to embodiments of the invention. The Data AgglomerationModule 104 associated with the DW 102 retrieves snapshots of raw salesrecords/transactions 112 and transcribes the numerical and quantitativedata of the records with an attached timestamp at regular intervals. Inaddition, the records 112 may be accumulated per sales representativeand per that representative's management hierarchy. During this processa number of calculations can be made, including but not limited to,standard ‘sales metrics’ like the percentage of a representative's (orgroup of representatives') sales quota they have attained as of a givendate. A data cache and metric score calculated by the Data Agglomerationcan be stored back on the DW 102.

One task of data agglomeration module 104 is to retrieve sales records(sales transactions) 112 and cache a subset of those records, thequantitative data, with a timestamp. The module also can cache datausing data transformations as well as using data models, and heuristicsin combination with previous data caches to produce various salesmetrics. One of ordinary skill in the art can easily determine that theData Agglomeration module 104 could be performed in a number ofdifferent ways and similarly that the scope of the inventive subjectmatter would not be diminished by not using this module. Rather theinclusion of this module is an efficiency enhancement to the novelaspects of the current invention. Similarly, the caching andagglomeration can occur in memory, on disk, or in any other method whichshould be obvious to one of ordinary skill in the art.

FIG. 3 is an illustration of a data query module 106 according toembodiments of the invention. The Query Module 106, also referred to asa Data Query and Display Module (DQ-DM) associated with the DW 102allows the user initiation of queries and displays the quantitative datagraphically in combination with a user's group and other peer groups.Each metric has a customizable graphical display and custom metrics canbe created for display. These graphical metrics can be combined withother graphical and tabular data to create a ‘dashboard’ for salespeople.

The basic data flow is driven by a user request for data via display anduser interface 116. A query is generated, and the response data is sentto the query module 106 in combinations with various other elements ofthe DW 102. Next, the data and elements move to the run-time enginewhere transforms, heuristics and data models 118 are applied. Theresulting retrieved and calculated data is then provided to displaycomponents of display and user interface 116 for viewing by the user.

FIG. 4 is a machine learning and forecasting functional flow accordingto embodiments of the invention. The machine learning and forecasting(MLF) module 108 associated with the DW 102 is a data mining module usedwithin embodiments of the present invention. It takes input from thevarious sales data tables 112, user input heuristics, and user inputidealized data models. The output is generally various scores on eachmetric or combinations of metric as well as learned data models.

The MLF module 108 is comprised of a variety of components. Eachcomponent contains a collection of algorithms that can be applied bothin parallel and sequentially to produce output within a component. Thefollowing components are useful in the preferred embodiment, but onewith ordinary skill in the art can easily see that not all of thecomponents are necessary, for example, either or both of the heuristicsengines 404 and 408 could be eliminated without altering the intendedfunctionality of the current invention.

MLF:

-   -   Data Manipulation Algorithms (DMA) 402 (further described in        FIG. 12)    -   Pre-DMF Heuristics Engine (HE1) 404 (further described in FIG.        14)    -   Data Mining and Forecasting Algorithms (DMF) 406 (further        described in FIG. 13)    -   Post-DMF Heuristics Engine (HE2) 408 (further described in FIG.        14)

The process flow illustrated in FIG. 4 is as follows:

-   -   Sales records 112 and elements of the DW 102 are retrieved and        input to the DMA 402 which is a component of MLF 108. The        configuration settings specify what algorithms are applied in        combination with DW 102 elements.    -   The output of the DMA 402 is passed to HE1 404 which is also a        component of MLF 108 along with retrieved heuristics from the        DW. Various heuristics specified by the configuration settings        are applied to the input.    -   The output of HE1 404 is passed to DMF 406 where configuration        settings govern the algorithms used in what combination with DW        102 elements.    -   The output of the DMF 406 is passed to the HE2 408. Internals        proceed similar to step 2.    -   The output of the HE2 408 is sent to the DW 102. The output may        be some combination metric-scores, derived data store, learned        data models and learned heuristics.

FIG. 5 is a future planning module functional flow according toembodiments of the invention. The Future Planning and Analysis module(FPA) 110 illustrated in FIG. 5 allows users to play out futurescenarios and use learned and idealized models as well as current andpast data to execute these scenarios. The FPA 110 module builds uponprevious modules capabilities and includes a scenario analysis engine502 that adds the ability to store and retrieve ‘scenarios’ and executethese scenarios in isolation in addition to changing base data and modelvalues for a particular scenario. Scenario analysis engine 502 interactswith MLF 108 based on input received from a user interface 114. Theresults of the scenario may be displayed 504 via user interface 114.

Examples 6 and 7 discussed below provide example use cases of the FPA110.

FIGS. 6 and 7 are diagrams illustrating the revenue growth MLF examples.The MLF module 108 is used to do the main data analysis of the FPA 110.The module is supplied with previous inputs from the data warehouse 102and sales records 112 as well as any ‘stored scenarios’ in the DW. TheUser interface 116 of the FPA module 110 allows the user to manipulateany data point that has been loaded from the DW 102 as well as changeparameters of the various ‘models’ used in the MLF module 108. Thechanges entered by the UI are updated into the MLF via the ‘ScenarioAnalysis Engine’ 502, a sub-component of the FPA 110 that is shown inFIG. 5.

In addition to accepting data and model changes from the UI, the‘Scenario Analysis Engine’ 502 contains a specific script, or algorithm,that specifies what components of the MLF 108 are used and in whatsequence to produce a desired output analysis 604. Various analyticalmethods 602, such as time series analysis or smoothing methods may beapplied. Output analysis 604 may include an idealized averageperformance profile or and idealized top-performance profile.

FIGS. 1-9 have illustrated various aspects of the system and systemoperation. Further details on the operation of the system are providedbelow in FIGS. 15-19. As noted above, MLF 108 may use various algorithmsduring its operation. Further details on these algorithms are providedin FIGS. 12-14.

FIG. 12 provides further details regarding data manipulation algorithms402. The Transforms section 1206 lists several example transforms whichmay apply in some embodiments. One of ordinary skill in the art canrecognize that this set of examples is representative and notexhaustive. Similarly, this step might use any number or combination ofthese transforms without altering the scope or intent of this patent.

FIG. 13 is a flow chart of the Data Mining and Forecasting Algorithms.Example Algorithms 1306 for FIG. 13 include, but are not limited to thefollowing:

-   -   Bayesian Classifier    -   Perceptron Algorithm    -   Expectation Maximization    -   K Nearest Neighbor    -   Radial Basis Function Networks    -   Relaxation Discriminate Classification    -   Divisive Clustering Algorithms    -   Agglomerative Clustering Algorithms    -   Ho-Kashyap Procedures    -   Linear Programming    -   Moving Average Forecasting (Family of Algorithms)    -   Box-Jenkins Forecasting    -   Holt-Winters Seasonality Forecasting    -   Wavelet Forecasting    -   Iterative Optimization Algorithms    -   Evolutionary Optimization Algorithms    -   Theoretical Queuing Models

In FIG. 13, the list of Algorithms 1304 (1 through N) represents anycombination of data mining and forecasting algorithms. The system mayuse a single algorithm or any number of multiple algorithms inconjunction. These algorithms could include any commonly known datamining algorithms such as clustering, outlier detection, linearregression, or any other technique known to someone with ordinary skillin the art. The section labeled “Ensemble Methods” 1306 lists severaltechniques used to resolve the combination of the data miningalgorithms. One of ordinary skill in the art can recognize that thislist is non-exhaustive and that any of the listed approaches alone or incombination could be used to achieve the desired purpose.

In FIG. 14, the section labeled “Apply Rule” 1404 represents any type ofrule that would be available in a rules engine or expert system, asknown by one of ordinary skill in the art. Examples include, “Deal takeslonger than 30 days AND deal size >$10,000” or “Quota attainment >50%”or “Sales funnel stage <3” or “Sales person progress score >B+” or anyother construction that proves useful to aiding the sales process.

FIGS. 15-19 are flowcharts illustrating methods according to variousembodiments of the invention. The methods to be performed by theembodiments may constitute computer programs made up ofcomputer-executable instructions. Describing the methods by reference toa flowchart enables one skilled in the art to develop such programsincluding such instructions to carry out the method on suitableprocessors (the processor or processors of the computer executing theinstructions from computer-readable media such as ROM, RAM, CD-ROM,DVD-ROM, flash memories, hard drives and other media). The methodsillustrated in FIGS. 15-19 are inclusive of acts that may be taken by anoperating environment executing one or more exemplary embodiments of theinvention.

FIGS. 15A and 15B are a flowchart illustrating a Pipeline Analysismethod according to embodiments of the invention. Although the examplemethod includes details regarding how the modules interact with eachother to provide a specific analysis in particular embodiments, itshould be noted that operations in the method may be performed bydifferent modules in various alternative embodiments.

In some embodiments, the method used input data comprising a list ofsales periods to analyze and a sales strategy to analyze. The method atbegins at block 1502 by extracting stage descriptive information fromdata warehouse 102 for the input sales strategy. At block 1504, a goalmodel for a given sales-strategy, sales-period pair is extracted. Atblock 1506, fuzzy logic quality and priority heuristics for the givensales-strategy and period are extracted.

One or more of the actions in blocks 1508-1514 may be executed for eachsales period. At block 1508, revenue goals for the given sales periodare extracted. At block 1510, previous learned sales period models forthe given sales strategy, if any, are extracted from data warehouse 102.At block 1512, the data warehouse 102 is queried for sales transactiondata for the given sales period. In some embodiments, the salestransaction data may include ongoing, closed, lost or deferredtransactions. At block 1514, a seasonality model for the given salesperiod is extracted from the data warehouse.

The actions in blocks 1520-1528 may be executed for each salestransaction obtained at block 1512 in order to perform data cleaning,data smoothing and/or heuristic classification. At block 1520, asmoothing method may be used to fill in missing data. At block 1522,erroneous or biasing data points may be determined and excluded. Atblock 1524, sales strategy stage transition times are calculated foreach stage. In some embodiments, the DMA module performs the smoothing,exclusion of erroneous or biasing data points and/or calculation of thesales strategy transition times.

At block 1526, a quality score is classified according to a qualityheuristic. At block 1528, a priority score is classified according to apriority heuristic. In some embodiments, the quality score and thepriority score may be classified by a Pre-HE module.

The actions in blocks 1530-1538 may be part of a loop that is executedfor each sales period in the input sales period list. Further, block1530-1536 may be part of a loop that is executed for each sales strategystage. At block 1530, weighted transition times for all salestransaction are computed. At block 1532, weighted abandonment times forall sales transactions are computed. At block 1534, the transition andabandonment times are classified according to the current goal model. Atblock 1536, a M/M/s or other Queuing Model is built for the currentstage.

At block 1538, the M/M/s Queuing Models for each stage is assembled intoa multi-level queuing network.

At block 1540, a M/M/s Queuing Network goal is built using the goalmodel. At block 1542, Queuing Network ‘Previous Data’ is built from aprevious learned model.

Blocks 1544-1550 are repeated for each learning algorithm, and for ‘n’time steps within each learning algorithm. At block 1544, a random salestransaction is selected. At block 1546, if the randomly selected salestransaction fits the selected ‘Goal’ model, then the ‘Goal’ model weightis reinforced. At block 1548, if the randomly selected sales transactionfits the selected ‘Previous Data’ model, then a ‘Previous Data’ modelweight is reinforced. At block 1550, if the randomly selected salestransaction fits the ‘Current Data’ model, the a ‘Current Data’ modelweight is reinforced.

At block 1552, an aggregate model is formed by combining each level ofthe ‘Goal’, ‘Previous Data’ and ‘Current Data’ models using the learnedweights.

Blocks 1554-1558 may be executed for each sales transaction. If theaggregate model accurately predicts the outcome of the sales transaction(block 1554) and the transaction has a high quality score (block 1556),then a positive metric score is assigned for the transaction (block1558).

After the sales transactions have been evaluated, a post-HE module mayoptionally be executed at block 1560.

At block 1562 the learned metric scores are stored in data warehouse102. At block 1564, the aggregate module is stored as the ‘PreviousData’ in data warehouse 102.

See Example 3a below for a description of an example application of thePipeline Analysis method illustrated in FIGS. 15A and 15B.

FIGS. 16A and 16B are flowcharts illustrating a revenue forecastingmethod according to embodiments of the invention. In some embodiments, alist of sales periods to analyze and a list of sales people to analyzemay be supplied as input for the method. A pseudo code description ofthe method is as follows:

 Extract ‘Goal Model’ for given Sales-Periods (block 1602)  ExtractFuzzy Logic Quality and Priority Heuristics for given Sales-Period(block 1604)  For Each Sales-Period (blocks 1606 1612)   Extract“Revenue Goals” for given Sales-Period (block 1606)   Extract previouslearned Sales-Period models for given Sales-Strategy, if any. (block1068)   Query Sales Transaction Data for Sales-Period {Ongoing, Closed,Lost, Deferred} (block 1610)   Extract Seasonality Model for givenSales-Period (block 1612)  For Each Sales Transaction (blocks 1620-1628)  Use smoothing method to fill in missing data (DMA Module, block 1620)  Exclude erroneous or biasing data points (DMA Module, block 1622)  Calculate Revenue Goal Attainment (DMA Module, block 1624)   ClassifyQuality Score according to Quality Heuristic (Pre-HE, block 1628)  Accumulate data and Scores to the sales-person-subject of thetransaction (block 1630)  For Each Sales-Person (block 1630)   For EachForecasting Algorithm    Calculate Forecast of remaining time units incurrent Sales period given past data (Block 1630)  Quality score theforecasts according to Fuzzy Logic Quality Heuristics (block 1640) Exclude erroneous or biasing forecasted data(block 1642)  RecordConsensus, High and Low forecasts for each time-division-unit. (block1644)  Store all Forecasted values and Scores in Data Warehouse(block1650)

FIGS. 17A and 17B are a flowchart illustrating a method for scoringperformance according to embodiments of the invention. A list of salesperiods to analyze, a list of sales people to analyze, and/or a list ofsales groups to analyze may be provided as input to the method in someembodiments. A pseudo code description of the method is as follows:

 Extract ‘Goal Model’ for given Sales-Periods (Block 1702)  ExtractFuzzy Logic Quality and Priority Heuristics for givenSales-Periods(Block 1704)  For Each Sales-Period (Blocks 1706-1718) Extract ‘Revenue Goals’ for given Sales-Period(Block 1706)  ExtractLearned Revenue Profiles for given Sales-Period and allPersons/Groups(Block 1708)  Extract Idealized Revenue Profiles for givenSales-Period and all Persons/Groups(Block 1710)  Extract LearnedPipeline Analysis data for given People/Groups(Block 1712)  Extractprevious learned Sales-Pipeline models for given Sales-Strategy, ifany(Block 1714)  Query Sales Transaction Data for Sales-Period {Ongoing,Closed, Lost, Deferred}(Block 1716)  Extract Seasonality Model for givenSales-Period(Block 1718) //Data Cleaning, Data Smoothing and HeuristicClassification For Each Sales Transaction (Blocks 1720-1726)  Usesmoothing method to fill in missing data (Block 1720)  Exclude erroneousor biasing data points (Block 1722)  Classify Quality Score according toQuality Heuristic (Block 1724)  Accumulate data and Scores to thesales-person-subject of the transaction (Block 1726) //Data Analysis(DMA Module) For Each Sales-Person (Blocks 1730-1738)  For EachSales-Period (Blocks 1730-1738)   Calculate Revenue Goal Attainment(Block 1730)   Classify as {Top, Average, Low} Performer (Block 1732)  Calculate Revenue Variance (Block 1734)   Calculate difference betweenLearned Model and Actual data (Block 1736)   Calculate differencebetween Idealized Model and Actual data (Block 1738) //(DMF Module) ForEach Learning Algorithm (Blocks 1740-1746)  For n time steps (Blocks1740-144)   Select Random Sales Person/Group (Block 1740)   IfTransaction Fits Idealized Model Then reinforce Idealized Model weight(Block 1742)   If Transaction Fits Learned Model Then reinforce LearnedModel weight (Block 1744)  Form Aggregate Model based upon Idealized andLearned Models (and weights) (Block 1746) //Post-HE module For EachSales-Person and Group  Reclassify Person/Group according to differencebetween Learned and Idealized Model as {Top  Average, Low} Performer(Block 1750)

Store all Learned Models, Scores and Classifications in Data Warehouse.(Block 1760)

FIG. 18 is a flowchart illustrating a method for allocation of quotas bysales periods according to embodiments of the invention. A list of salesperiods to analyze, a list of sales people to analyze, and/or a list ofsales groups to analyze may be provided as input to the method in someembodiments. A pseudo code description of the method is as follows:

1. // Query Module 2. Extract Revenue Models for given Sales-Periods(Block 1802) 3. Extract Pipeline Models for given Sales-Periods (Block1804) 4. Extract Fuzzy Logic Quality Heuristics for given Sales-Periods(Block 1806) 5. For Each Sales-Period 6.    Extract ‘Revenue Goals’ forgiven Sales-Period (Block 1808) 7. 8. 9. For Each Sales Period (Blocks1810-1844) 10.   // Heuristic Classification 11.   Initialize Quotas Setfor Each Person/Group Based upon current Sales Period (Block 1810) 12.  Accumulate Quotas Set to calculate a Total Quota (Block 1812) 13.  Classify Quality Score for each Quota according to Quality Heuristic(Block 1814) 14.   Classify Quality Score for Total Quota vs. Goal Quotaaccording to Quality Heuristic (Block 1816) 15. 16.   Best-Quotas =Quotas //Each of these is a set of individual Quotas (Block 1818) 17.18.  For n time steps (Blocks 1822 36) 19.     Quotas-New = Quotas(Block 1822) 20.     Select Random Person/Group Quota from Quotas-New(Block 1824) 21.     Randomly adjust selected Person/Groups Quota amoderate amount via White-noise function (Block 1826) 22.     ClassifyQuality Score for this new Quota according to Quality Heuristic (Block1828) 23.     Classify Quality Score for Total Quota vs. Quotas-Newaccording to Quality Heuristic (Block 1830) 24.     If Quality Score ofQuotas-New ≧Best-Quotas (Block 1832) 25.      Best-Quotas = Quotas-New(Block 1834) 26.     Else If Best-Quotas is unchanged for m time-steps(Block 1836) 27.      Add Best-Quotas and its Quality Score to ResultQueue (Block 1838) 28.      n = 1 (Block 1840) 29.      Goto 11 (go toBlock 1810) 30. 31. Select a Quotas Set from Result Queue with the bestQuality Score (Block 1842) 32. Store this Quotas Set For the given SalesPeriod in Data Warehouse (Block 1844) 33. 34. //Quality Score of a Quotais a function of its Forecasted Revenue and Pipeline Performance vs. the35. // overall Goals

FIG. 19 is a flowchart illustrating a method for allocation of quotas byterritories according to embodiments of the invention. A list of salesperiods to analyze, a list of sales people to analyze, and/or a list ofsales groups to analyze may be provided as input to the method in someembodiments. A pseudo code description of the method is as

1. Extract Revenue Models for given Sales-Territories (Block 1902) 2.Extract Pipeline Models for given Sales-Territories (Block 1904) 3.Extract Fuzzy Logic Quality Heuristics for given Sales-Territories(Block 1906) 4. For Each Sales-Territories 5.     Extract ‘RevenueGoals’ for given Sales-Territories (Block 1908) 6. 7. 8. For Each SalesTerritories (Blocks 1910-1942) 9.   // Heuristic Classification 10.  Initialize Quotas Set for Each Person/Group Based upon current SalesTerritories (Block 1910) 11.   Accumulate Quotas Set to calculate aTotal Quota (Block 1912) 12.   Classify Quality Score for each Quotaaccording to Quality Heuristic (Block 1914) 13.   Classify Quality Scorefor Total Quota vs. Goal Quota according to Quality Heuristic (Block1916) 14. 15.   Best-Quotas = Quotas //Each of these is a set ofindividual Quotas (Block 1918) 16. 17.   For n time steps (Blocks 19201934) 18.    Quotas-New = Quotas (Block 1920) 19.    Select RandomPerson/Group Quota from Quotas-New (Block 1922) 20.    Randomly adjustselected Person/Groups Quota a moderate amount via White-noise function(Block 1924) 21.    Classify Quality Score for this new Quota accordingto Quality Heuristic (Block 1926) 22.    Classify Quality Score forTotal Quota vs. Quotas-New according to Quality Heuristic (Block 1928)23.    If Quality Score of Quotas-New ≧Best-Quotas (Block 1930) 24.    Best-Quotas = Quotas-New 25.    Else If Best-Quotas is unchanged for mtime-steps (Block 1932) 26.     Add Best-Quotas and its Quality Score toResult Queue (Block 1934) 27.     n = 1 (Block 1936) 28.     Goto 11 (goto Block 1910 at Block 1938) 29. 30.   Select a Quotas Set from ResultQueue with the best Quality Score (Block 1940) 31.   Store this QuotasSet For the given Sales Territories in Data Warehouse (Block 1942) 32.33. //Quality Score of a Quota is a function of its Forecasted Revenueand Pipeline Performance vs. the 34. // overall Goals

EXAMPLES

A series of examples illustrating the operation of the above-describedsystems and methods will now be provided.

Example 1 Forecasting Revenue

Using the data query and display module, sales data is retrievedrepresenting some window-in-time of sales revenue as well as ongoingin-process sales records. From the DW 102 a configuration is retrieveddefining the data flow and data transformed to be used. Also from the DW102 a cache of data from the previous run(s) of the data caching andagglomeration module, metric scores and derived data from previous runsof this module is fetched.

The Data Manipulation Algorithm (DMA) module 402 (FIGS. 4 and 12) isapplied to the raw sales data to align it with records from the ‘datacache’ using data transforms. Supplementary calculations are also madeto the metric scores. An example output data stream from the modulewould be a large list of quantitative data associated with sales-people,sales-periods, products sold and performance metrics for each of theseunits.

The output of this module is fed to the HE1 404. One example function ofthe HE1 module 404 is to clean the data of statistical outliers beforepassing to the DMF module 406. Another example is to create additionalquantitative metrics. An example of this is to assign a heuristiccategory to individual sales people. Sales people who have consistentlymet or exceeded goals (as defined by evaluating any number of factors)for a large percentage of sales periods would be marked as ‘topperformers’. People who meet most goals for the same time period wouldbe marked as ‘average performers’, the rest as ‘under-performing’. Athird example would flag particular metrics as having an anomaly withoutnecessarily excluding it, such as a sales person who had his or herperformance enhanced or degraded by an anomalous event and this shouldnot unduly influence later calculations.

The next sub-module in the chain is the code DMF 406 (FIGS. 4 and 13).This module uses the data and metric stream as an attribute set in thecombination of data models with a suite of forecasting algorithms toestablish a set of revenue forecasts. An example data model would be auser-entered ideal profile of how a sales person or group's variousmetrics should be shaped numerically. It may contain various weights forhow individual metrics would be used in forecasting algorithms. Inaddition other parts of the DMF module (FIGS. 4 and 13) could be appliedto learn a data model, or profile of how attribute values progress intime associated with performance levels, sales groups and products. Thisengine also is able to learn new data models with various algorithms byexamining previous data in a posteriori plus feedback fashion.

The output of the DMF module 406 flows to the HE2 sub-module 408 (FIGS.4 and 14). In this scenario HE2 408 would be used to take the suite ofrevenue forecasts and assign quality scores based upon both user enteredheuristics and learned heuristics. This engine also is able to learn newheuristics with various algorithms by examining previous data in aposteriori plus feedback fashion. The final output of the examplescenario is a set of revenue forecasts for each person, group andaggregate forecasts for sales. In addition accuracy scores for eachforecast, as well as learned data models and heuristics are stored.

Example 2 Scoring the Performance of Sales People and Groups

Similar to Example 1 with the modification that all calculations aredone to score the performance of individual people and groups withrespect to learned data models of top-performers as well as with respectto user-entered ideal models of performance. An example of this would bean ideal revenue growth of a sales person over time. A second example isa relative analysis of the person's ongoing sales ‘pipeline’ withrespect to learned and ideal models of good performance. In either casethe system can learn the historical sales models for an individual orhigh performing sales representative using any of a variety of machinelearning techniques. In addition, an idealized model can be entered bythe sales manager. The resulting comparison can aid in the understandingof how individuals relate to one another and to an idealized individual.This helps sales representatives understand their relative performance,and it helps sales managers understand if there is a general deviationbetween their idealized model and the actual performance of anindividual or group.

Example 2 Details Ideal Revenue Growth or Progression of a Sales PersonOver Past Time and into the Future

This analysis occurs in the MLF and DQ-DM Modules. See FIGS. 6 & 7 fordiagrammatic details. The method's purpose is to extract raw data andlearn a central pattern for revenue progression in time of a salesperson or group that is relative to a combination of an idealizedpattern and the Revenue Progression pattern of the group of TopPerformers.

MLF Module Operation for Example 2:

The sales data for all persons groups is queried from the ‘SalesRecords’ and from the Data Warehouse. Also from the Data Warehouse thevarious heuristics are retrieved. The Sales records consist of the‘Closed, Lost and Deferred’ Sales Revenue for a range of hierarchicalsales periods over the past X periods in time. This data is extractedper person and per group (aggregate). Individuals in the Top-Performergroup are included in this query with their membership in this grouptagged as such. The following heuristics are retrieved from the DW. Notethat these examples are not exhaustive and one of ordinary skill in theart can easily add to this list.

-   -   MLF-Module Pre-Heuristic-Engine Examples        -   (Learned) Sales Revenue Variance: Number of sales periods            where the revenue variance of an individual customer (or            class or group of customers) have variance below some            percentage threshold. Variance is the percentage drop or            gain compared to last period.        -   (Idealized) Sales Revenue Variance Profile:        -   The ideal profile of revenue variance. This is characterized            by a range of growth or decrease percentages            period-over-period. An example would be that ideal growth or            expected decrease in sales period-over-period should vary            between −10% and +15% in the 3^(rd) month of a given year            vs. the 2^(nd) month.        -   (Learned) Sales Revenue Profile of Top Performers:        -   Form an aggregate profile of the progression of the revenue            profile for all ‘Top Performers’. This profile would be            normalized to be a numerical curve of percentage increases            starting from the initial revenue goal of the starting sales            period of the analysis. The profile would be weighted by the            overall ‘performance score’ of the individual. The basic            computation is a smoothed weighted average using any            combination of classical methods (iterative exponential            smoothing) or newer time-series analysis methods with or            without seasonality adjustments.        -   (Learned) Sales Revenue Profile of Sales Group:        -   Same as the previous analysis but using the            subject-salesperson's peer-group.        -   (Learned) Sales Revenue Profile of Sales Person:        -   Same as previous analysis except done for each individual in            isolation.        -   (Idealized) Sales Revenue Profile:        -   This is an idealized profile established & input by sales            managers or executives that gives an ideal average and            spread of a revenue curve over time.

Main DMF Module Details:

In this step previous learned profiles and new sales records are used tolearn updated profiles. This may be accomplished by running a suite ofclassical forecasting methods, time series analysis methods, smoothingmethods and seasonality adjustments. The outputs of the individualanalysis methods are aggregated via a weighted scheme. Note that theweights of each method are learned as well. After a run is performed andforecasts are stored, they are used in the next run and compared to theactual reported revenue gained, lost, or otherwise deferred. Theaggregation weights of the different methods are adjusted based upon thesuccess or failure of the particular method's predicted data vs. actualdata.

The output of the DMF module (FIG. 13) is processed by the MLFPost-Heuristics engine 408 (FIG. 14). Here each learned profile is givena quality score that is a function of the accuracy of the predictionsfrom the DMF module and it's similarity to the idealized models. Inparticular embodiments, the heuristic analysis is a set of fuzzy logicrules.

Output of MLF Module:

-   -   Quality score for each person or group    -   Updated ‘Learned’ profiles.    -   Updated metadata for Idealized model-profiles.

DQ-DM Module for Example 2:

Given a ‘subject’ (sales representative or group) of analysis, thefollowing is performed to display to the user a graded analysis of thesubject's revenue growth or progression. See FIGS. 7 and 10.

From the DW the following can be retrieved:

-   -   Run-Time profile comparison and analysis methods    -   Learned Revenue Profile for the Subject    -   Learned Revenue Profile for the Subject's Group    -   Learned Revenue Profile for the Top Performers    -   Idealized Revenue Profile.    -   Learned Revenue Variance Profile for the Subject    -   Learned Revenue Variance Profile for the Subject's Group    -   Learned Revenue Variance Profile for the Top Performers    -   Idealized Revenue Variance Profile.

The run-time heuristics are performed on the above data. The Heuristicsproduce two types of output, comparative grading and absolute grading.The comparative grading assigns a quality score for the subjects profilerelative to a comparison-group. Typically this is the subjects peergroup in the organization and the group of top-performers. The absolutegrading is similar except that a score is assigned assessing how wellthe subjects profile fits the idealized profiles. FIG. 10 has an examplechart and assigned grade for an absolute revenue trend and profileanalysis. One of average skill in the art can easily see othervariations of this example.

Example 3 Scoring Sales Processes

Similar to examples 1 & 2. The module is applied to the problem ofevaluating an entire sales processor sales methodology with respect tothe ideal model of the process as well as compared to other goodperforming processes or methodologies. Note that this analysis can beindependent of individuals looking at the process in isolation. It canalso be used to identify weak individuals or groups that bring theoverall performance of the process or methodology down. The novelty ofthis approach is it's usage in CRM software applied to sales processanalysis and its combination with iterative machine learning methods.

Example 3a Ideal Pipeline Analysis of Sales Pipeline & Strategy

Background for Example 3a:

A sales pipeline is an abstract construct consisting of series of‘stages’. At each stage a set of tasks is to be performed by the salesrepresentative. For example, stage 1 might be called ‘Leads’ wherepotential deals are placed in this first stage upon initial contact withthe sales representative. To progress to Stage 2 (‘Demo Product’) thedeal's primary contact must exchange preliminary information with thesales representative and agree to schedule a product demonstration.Stage 7, for example, could be the ‘Negotiation’ phase whererepresentative and client agree to financial terms of a deal. The Set ofstages is generally referred to as a Sales Strategy. Representatives maybe working on potential clients using a plurality of Sales Strategies.

The active deals of a given representative can be assigned to a stage ina particular strategy. A representative's ‘raw pipeline’ is a set ofmonetary values that are the sum of the assumed value potential deals ina given stage. A representative's ‘forecasted pipeline’ is a set ofvalues as above except that each stage is assigned a conversion ratepercentage. The raw values in each stage are multiplied by theconversion rate percentage to form a set of monetary values that arepresentative terms his or her expected forecast. Ordinarily thesepercentages are manually assigned by users.

Sales Strategy and Pipeline Analysis ‘Learning Method’:

Various data is requested from the DW 102 and Sales Records 112. Arepresentative outline of this data is shown below:

-   -   Closed or Lost or Deferred Revenue History    -   Date and time of each stage transition for all deals    -   Sales Process stages and stage-forecast-percentages    -   Sales Goals    -   Ongoing Active Deals    -   Quality Heuristics        -   Revenue Model (see Example 2)        -   Strategy Quality Rules        -   Data Cleaning Procedures

The data above is loaded and a set of algorithms is applied to them toextract statistical information on the stage transition timings, stageabandonment rates (lost deals) and deferred or backward stage moves. Allsales strategies are modeled with a multi-level queuing model such asM/M/s queuing network (for further details, see An Introduction toStochastic Modeling, by Samuel Karlin, Howard M. Taylor, Academic Press,3 edition (February 1998), which is hereby incorporated by referenceherein for all purposes).

This analysis allows the predictive modeling of the sales strategy andcompares it to the user entered expected conversion rates from one stageto the next. This analysis also allows the modeling of how changinghuman resources allocated to specific stage will affect the behavior ofthe strategy as a whole.

This queuing network method is combined with a set of fuzzy logic rulesets, for example, that classify deals based upon their characteristics.Specifically the attributes of a deal are used to perform a supervisedmachine learning algorithm based upon the outputs of the queuing modeland the system learns a classification system for assigning qualityscores to deals. These scores are used to prioritize attention to dealsof various types. Deals that the classifier system and queuing modelpredict will transition quickly between stages are given an increased‘forecast percentage’. While deals with some attribute that the MLalgorithm has identified as correlated with slow progress will beassigned a lower forecast percentage.

Note also that this analysis allows prediction of deal close times andassigning scoring metrics to individual sales representatives and groupsbased upon how a particular deal is progressing (for example—being 2days past typical transition times in stage 2 indicates that thepredicted close date will be 10 days behind typical). Another algorithmin this suite is the forecasting, smoothing and time series analysisalgorithms mentioned in example 2. Here these techniques can be used toboth repair missing data and to provide compatible sales strategyforecasts for comparative analysis and decisioning. It is also used toprovide seasonality adjustments to the output of the queuing network.

This system can be run at a frequency relevant to the typical businesscycle of a sales strategy. For example, if a typical deal is closed in30 days and some component of a deal is likely to change every day, thenthe system would be run at least twice a day. The data and models builtfrom the last run are loaded and their predictions checked. The variousquality weights on the output of each algorithm are adjusted by areinforcement learning algorithm.

Sales Strategy and Pipeline Analysis ‘Query, Analytic Comparison andDisplay Method’:

FIG. 9 is an illustration of this process. It is similar to FIG. 7. Theaggregate outputs of the MLF module for this task are loaded along withthe subject sales process. An example output is given on FIG. 11.

The basic method is shared with Example 2's DQ-DM section along withFIG. 7.

This following data can be loaded:

From the DW the following can be retrieved:

-   -   Run-Time profile of the current sales strategy.    -   Learned forecast stage percentages    -   Quality Metrics for poor, average and excellent performing deals    -   Seasonality Model    -   Data associated with the subject sales strategy    -   Sales records for open deals within this strategy

The output of this model is an aggregated summary of the open deals inthe current sales period for a given sales period. Each stage in thestrategy is assigned a score for overall execution and a forecastedoverall conversion rate. FIG. 15 described above provides an examplealgorithm that details how the modules interact with each other toprovide this (Example 3a) analysis.

Example 3b Ideal Pipeline Analysis of Sales Representative or Group

Sales Strategy and Pipeline Analysis ‘Learning Method’:

-   -   Example 3b is similar to Example 3a, except that the analysis is        done for a specific sales representative or group across the        range of sales strategies they work on.

Sales Strategy or Pipeline Analysis ‘Query, Analytic Comparison andDisplay Method’:

-   -   See Example 3a except that this analysis is done for a specific        sales representative or group across the range of sales        strategies they work on.

Example 4 Allocation of Quotas

The system can be used to allocate or re-allocate sales goals or quotas.Using a suite of machine learning or optimization algorithm one canlearn optimal allocation of quotas such that some fitness function canbe maximized. The fitness function would be comprised of a weighted sumof factors and use the outputs of Example 2a as primary inputs to thefitness function to reallocate quotas.

Example 5 Allocation of Territories

The system can be used to allocate or re-allocate sales territories. Forexample, using a suite of machine learning or optimization algorithm onecan learn optimal allocation of quotas such that some fitness functionis maximized. The fitness function may be comprised of a weighted sum offactors and use the outputs of Example 2a as primary inputs to thefitness function to reallocate territories.

The FP module 110 as supported by the DW 102 allows sales people toexecute ‘what if’ scenarios for planning various organization attributesfor the future. The basic idea is to solve (by isolated analysis,approximation, or by a direct technique)(in spirit, not necessarilyliteral solving) for a particular variable. In basic algebra when onesolves equations (s), some number of variables are chosen as ‘unknown’while the rest of the variables that are known are used to identifyvalues and ranges of values for the unknown variables that fulfill theequations.

The input to the FP 110 can be made up of the raw sales record data 112as well as the DW 102 using all previously mentioned elements and anadditional ‘scenarios’ element. The processing sub-modules are the MLF108, a scenario analysis engine (SAE) 502 and a display and userinterface element 504. A display and user interface components allow theuser to see the output of the FP 110 as well as interact with somenumber of quantitative attributes. Additionally, the SAE 502 may containspecific processes as described herein.

The display 504 and user interface element 116 can communicatebi-directionally with the SAE 502 & MLF 108. The FP 110 can also storescenarios that users have created during the use of the FP 110. Onepurpose of the SAE 502 and MLF 108 is to use ‘conditional’ data tocreate new MLF analysis outputs given this conditional data. Theoutputs, conditional data, and associated elements of the DW 102 aretermed a ‘scenario’ and also stored in an area of the DW 102. Thescenario engine 502 contains a number of algorithms as described herein.

Example 6 Goal Based Revenue Planning

A manager has in mind a particular goal for a given metric, in this casegoal-revenue for some number of time periods into the future. However,after looking at revenue forecasts from the MLF it is apparent thatrevenue forecasts are short of the goal. At this point a manager has anumber of business factors that can be adjusted. Sales people'sindividual goals can be altered, product pricing can be adjusted and newsales people can be hired. As an example, a manager may use the systemto determine which of these factors should be altered to best meet thedesired goal.

For example, using the FP 110, a manager can input the desiredgoal-revenue and look at how factors can be altered to meet this goal.Taking one factor, as an example the individual revenue goals, thesystem can be asked to meet the new revenue goal by assigning newindividual goals. The output here would be a breakdown of goal changes,as well as a likelihood score of the users meeting the new goals givenpast history and current status. It would also output an overalllikelihood score of meeting the new goal-revenue.

Example 7 Hiring and Firing of Individuals

Building on Example 1 of the FP 110, the manager can also attempt tomeet some new goal by examining the effect of hiring additionalindividuals plus firing and/or replacing existing individuals. Themanger is presented with a list of individuals, their past metrics,future goals and future forecasts. Each individual is also assignedvarious performance scores. The manger can in the simplest caseeliminate individuals from the list and look at the effect on the futuregoals, forecasts and likelihood numbers. Another action would be to addan individual to the list of some performance level, presumable either‘average’ or ‘top’ and look at the effect on forecasts, likelihood andgoals. The third action would be the combination of the two, replacingan individual with a new individual.

A slightly more complex embodiment is to set up a schedule of hiring ofnew people at a given performance level to see how the forecasts areaffected over longer time scales. This staged hiring model would benefitfrom the accurate historical models of how long it took existing salespeople to come up to their current level of performance and similarlyindicate the growth rate for individual performance as they learn theparticulars of the current product and sales environment.

An embodiment of the heuristics would be both simple rule-sets as wellas fuzzy rule sets to assign grades or scores to individual metrics. Anembodiment of a data-model is a set of values describing how givenmetrics change over some other variable (an example would be a metricover time).

Thus as is evident from the above, the various embodiments may provide aRevenue Forecasting system to learn to forecast revenue as a combinationof standard approaches as well as ones based upon learning a profile ofthe members of the organization and their historical attributes,performance classification etc.

Further, the various embodiments may provide for scoring the performanceof Sales People and Groups, including:

-   -   Scoring a person/groups learned profile against known-good        profiles and entered ideal-profiles. Give detailed breakdown of        areas of difference    -   Scoring projected revenues vs. goals

Still further, the various embodiments may provide analysis ofperformance of Sales Strategy vs. Ideal/Goal, including:

-   -   Learning/building a model of Strategy-Pipeline based upon real        data    -   Comparing to Idealized model of Strategy-Pipeline    -   Allowing prediction of pipeline throughput per person group and        as a whole    -   Providing real-time scores per person group of deviation from        Ideal

Yet further, the various embodiments provide the ability to performFuture Planning and What-If Scenarios, including:

-   -   Goal based Revenue Planning    -   Given learned profiles of individuals and processes what is the        likelihood of meeting a particular Revenue Goal?    -   Breakdown reallocation of Quota increases based upon above        learned profiles    -   Breakdown reallocation of Territories based upon above learned        profiles    -   Hiring/Firing Planning—Similar to above with the addition of        allowing user to ask the system to evaluate the effect of adding        new people, replacing people, firing people and its effect on        future revenue and sales strategy performance

The various sales performance and forecasting system examples shownabove illustrate a novel predictive and profile learning salesautomation analytics system. A user of the present invention may chooseany of the above sales performance and forecasting system embodiments,or an equivalent thereof, depending upon the desired application. Inthis regard, it is recognized that various forms of the subjectinvention could be utilized without departing from the spirit and scopeof the present invention.

As is evident from the foregoing description, certain aspects of thepresent invention are not limited by the particular details of theexamples illustrated herein, and it is therefore contemplated that othermodifications and applications, or equivalents thereof, will occur tothose skilled in the art. It is accordingly intended that the claimsshall cover all such modifications and applications that do not departfrom the spirit and scope of the present invention. Therefore, it ismanifestly intended that this inventive subject matter be limited onlyby the following claims and equivalents thereof.

The Abstract is provided to comply with 37 C.F.R. §1.72(b) to allow thereader to quickly ascertain the nature and gist of the technicaldisclosure. The Abstract is submitted with the understanding that itwill not be used to limit the scope of the claims.

What is claimed is:
 1. A system comprising: a storage device adapted tostore one or more performance characteristics of one or morerevenue-generating personnel of an organization; a processing devicecommunicatively coupled to the storage device and adapted to: determineat least one first revenue forecasting parameter based on at least oneperformance characteristic of at least one person supporting generationof revenue for the organization; store the at least one first revenueforecasting parameter in the storage device; determine at least onesecond revenue forecasting parameter to forecast the revenue of anorganization, wherein the at least second forecasting parameter isdetermined at least in part from sales opportunities in a salespipeline; store the at least one second forecasting parameter in thestorage device; access the at least one first and at least one secondparameters; and forecast revenue of the organization using at least inpart the first and second forecasting parameters.
 2. A system accordingto claim 1 wherein the at least one performance characteristic useshistorical performance of the person.
 3. A system according to claim 1wherein the at least one performance characteristic uses theclassification of the person's capability to generate revenue for theorganization.
 4. A system according to claim 1 wherein the processingdevice is further adapted to: maintain a profile for the at least oneperson, wherein the profile includes at least one parameter indicativeof the capability to generate revenue.
 5. A system according to claim 1wherein the processing device is further adapted to determine thelikelihood of meeting a revenue goal using a learned profile of anindividual or group.
 6. A system according to claim 5 wherein theprocessing device is further adapted to reallocate sales quotas for theindividuals or groups based on the likelihood of meeting the revenuegoal.
 7. A system according to claim 5 wherein the processing device isfurther adapted to reallocate sales territories for the individuals orgroups based on the likelihood of meeting the revenue goal.
 8. A methodcomprising: determining at least one first revenue forecasting parameterbased on at least one performance characteristic of at least one personsupporting generation of revenue for the organization; storing the atleast one first revenue forecasting parameter; determining at least onesecond revenue forecasting parameter to forecast the revenue of anorganization, wherein the at least second forecasting parameter isdetermined at least in part from sales opportunities in a salespipeline; storing the at least one second forecasting parameter;accessing the at least one first and at least one second parameters; andforecasting revenue of the organization using at least in part the firstand second forecasting parameters.
 9. A method according to claim 8wherein the at least one performance characteristic uses historicalperformance of the person.
 10. A method according to claim 8 wherein theat least one performance characteristic uses the classification of theperson's capability to generate revenue for the organization.
 11. Amethod according to claim 8 further comprising maintaining a profile forthe at least one person, wherein the profile includes at least oneparameter indicative of the capability to generate revenue.
 12. A methodaccording to claim 8 further comprising determining the likelihood ofmeeting a revenue goal using a learned profile of an individual orgroup.
 13. A method according to claim 12 further comprisingreallocating sales quotas for the individuals or groups based on thelikelihood of meeting the revenue goal.
 14. A method according to claim12 further comprising reallocating sales territories for the individualsor groups based on the likelihood of meeting the revenue goal.
 15. Acomputer-readable memory comprising a set of instructions stored thereinwhich, when executed by a processor, cause the processor to perform aprocess comprising: determining at least one first revenue forecastingparameter based on at least one performance characteristic of at leastone person supporting generation of revenue for the organization;storing the at least one first revenue forecasting parameter;determining at least one second revenue forecasting parameter toforecast the revenue of an organization, wherein the at least secondforecasting parameter is determined at least in part from salesopportunities in a sales pipeline; storing the at least one secondforecasting parameter; accessing the at least one first and at least onesecond parameters; and forecasting revenue of the organization using atleast in part the first and second forecasting parameters.
 16. Acomputer-readable memory according to claim 15 wherein the at least oneperformance characteristic uses historical performance of the person.17. A computer-readable memory according to claim 15 wherein the atleast one performance characteristic uses the classification of theperson's capability to generate revenue for the organization.
 18. Acomputer-readable memory according to claim 15 further comprisingmaintaining a profile for the at least one person, wherein the profileincludes at least one parameter indicative of the capability to generaterevenue.
 19. A computer-readable memory according to claim 15 furthercomprising determining the likelihood of meeting a revenue goal using alearned profile of an individual or group.
 20. A computer-readablememory according to claim 19 further comprising reallocating salesquotas or territories for the individuals or groups based on thelikelihood of meeting the revenue goal.